Data input device

ABSTRACT

A data input device and method including an illuminator operative to illuminate at least one engagement plane by directing light along the at least one engagement plane, a two-dimensional imaging sensor viewing the at least one engagement plane from a location outside the at least one engagement plane for sensing light from the illuminator scattered by engagement of a data entry object with the at least one engagement plane and data entry processor receiving an output from the two-dimensional imaging sensor and providing a data entry input to utilization circuitry.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Divisional Application of U.S. patent applicationSer. No. 10/250,350, filed Dec. 18, 2003, which is the National Stage ofInternational Application No. PCT/IL01/01082, filed Nov. 26, 2001, whichclaims the benefit of U.S. Provisional application Ser. No. 60/260,436,filed Jan. 8, 2001; U.S. Provisional application Ser. No. 60/263,115,filed Jan. 19, 2001; U.S. provisional application Ser. No. 60/303,922,filed Jul. 6, 2001; and U.S. provisional application Ser. No. 60/338.365filed Nov. 2, 2001.

BACKGROUND OF THE INVENTION

The following patents and publications are believed to represent thecurrent state of the art:

Published PCT Application WO 01/59975A2; U.S. Pat. No. 6,266,048;Published European application EP 0 982 676 A1; Published Europeanapplication EP 1 039 365 Aw; U.S. Pat. No. 4,468,694; U.S. Pat. No.5,969,698; Published Japan application 2000029605; Published PCTapplication WO 00/39663; Published PCT application WO 01/54110 A1; U.S.Pat. Nos. 6,175,679; Published PCT application WO 99/13395 A1; U.S. Pat.Nos. 5,767,842; 6,043,805; 5,909,210; 5,786,810; 5,821,922; 5,864,334;Published PCT application WO 00/21024; U.S. Pat. No. 6,037,882; U.S.Pat. Nos. 6,121,960; 5,789,739; 6,031,519; 5,736,976.

SUMMARY OF THE INVENTION

The present invention relates to data input methods and apparatusgenerally.

There is thus provided in accordance with a preferred embodiment of thepresent invention a data input device including an illuminator operativeto illuminate at least one engagement plane by directing light along theengagement plane, a two-dimensional imaging sensor viewing theengagement plane from a location outside the engagement plane forsensing light from the illuminator scattered by engagement of a dataentry object with the engagement plane and a data entry processorreceiving an output from the two-dimensional imaging sensor andproviding a data entry input to utilization circuitry.

There is also provided in accordance with a preferred embodiment of thepresent invention a data input method, which included illuminating atleast one engagement plane by directing light along the engagementplane, employing a two-dimensional imaging sensor for viewing theengagement plane from a location outside the engagement plane forsensing light from the illumination scattered by engagement of a dataentry object with the engagement plane and receiving and processing anoutput from the two-dimensional imaging sensor and providing a dataentry input to utilization circuitry.

Further in accordance with a preferred embodiment of the presentinvention the data input device also included a data entry matrixprojector operative to project at least one visually sensible data entrymatrix onto a projection surface underlying the engagement plane.

Preferably, the visually sensible data entry matrix defines a keyboard.

Still further in accordance with a preferred embodiment of the presentinvention the illuminator includes an illuminator light source and aspatial light modulation element operative to receive light from theilluminator light source and to direct light along the engagement plane.

Additionally in accordance with a preferred embodiment of the presentinvention the projector includes a projector light source and a spatiallight modulation element which operates to receive light from theprojector light source and to project at least one visually sensibledata entry matrix onto a surface underlying the engagement plane.

Preferably, the spatial light modulation element includes a diffractiveoptical element.

Further in accordance with a preferred embodiment of the presentinvention the spatial light modulation element includes an asphericoptical element. Additionally or alternatively, the spatial lightmodulation element includes a joined double side truncated rod lensoptical element.

Typically the spatial light modulation element includes a transparency.

Further in accordance with a preferred embodiment of the presentinvention the two-dimensional imaging sensor includes a solid stateimaging sensor.

Still further in accordance with a preferred embodiment of the presentinvention the data entry processor correlates the output from thetwo-dimensional imaging sensor with the visually sensible data entrymatrix.

Additionally in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes a diffractive opticalelement which receives light from a diode laser via a collimating lens.

Typically the light passing through the diffractive optical element isreflected by a curved mirror having optical power via a lens onto theprojection surface.

Preferably, the diffractive optical element, the mirror and the lens areall integrally formed in a prism.

Further in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes an integrally formedbeam splitter an diffractive optical elements.

Preferably, in the data entry matrix projector, a beam of light from adiode laser passes through a collimating lens and impinges on twomutually angles surfaces of the beam splitter, which breaks the beam oflight into two beams, each of which passes through a separatediffractive optical element and impinges on the projection surface.

Typically the diffractive optical elements are integrally formed withthe beam splitter in a prism.

Further in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes a plurality ofdifferent diffractive optical elements, each of which typicallycorresponds to a different matrix configuration, which are selectablypositionable along a projection light path.

Still further in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes a diffractive opticalelement having a multiplicity of diffraction orders selected to providea matrix configuration which has a relatively low maximum diffractionangle.

Additionally or alternatively, the data entry matrix projector includesa diffractive optical element having a multiplicity of diffractionorders selected to provide a keyboard configuration, which has agenerally trapezoidal configuration.

Further in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes a diffractive opticalelement having a multiplicity of diffraction orders selected tocompensate for geometrical distortions inherent in the operation of thediffractive optical element, particularly at high diffraction angles.

Still further in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes a diffractive opticalelement having a multiplicity of diffraction orders selected tocompensate for geometrical distortions occasioned by a highly obliqueangle of projection.

Additionally in accordance with a preferred embodiment of the presentinvention in the data entry matrix projector, light from a pair of pointlight sources is combined by a beam combiner, such that tow light beamsemerge from the beam combiner and appear to originate in a singlevirtual light source positioned behind the beam combiner.

Preferably, the light beams pass through a shadow mask onto theprojection surface.

Further in accordance with a preferred embodiment of the presentinvention the data entry matrix projector includes an array of lightemitting elements and microlenses.

Typically, the light emitting elements are individually controllable.

Still further in accordance with a preferred embodiment of the presentinvention the data entry matrix project includes a monolithic pattern ofLEDs formed on a unitary substrate.

Further in accordance with a preferred embodiment of the presentinvention the two-dimensional imaging sensor is located on the oppositeside of a transparent engagement surface from the engagement plane,whereby the presence of the data entry object at the engagement planecauses light from the illuminator to be scattered and to pass throughthe transparent engagement surface so as to be detected by thetwo-dimensional imaging sensor.

Still further in accordance with a preferred embodiment of the presentinvention the data input device includes a transparent engagementsurface is coextensive with the engagement plane, whereby touchingengagement of the data entry object with the transparent engagementsurface causes light from the illuminator to be scattered and to passthrough the transparent engagement surface so as to be detected by thetwo-dimensional imaging sensor.

Preferably, the transparent engagement surface exhibits total internalreflection of a planar beam of light emitted by an illuminator andcoupled to an edge of the transparent engagement surface, wherebytouching engagement of the data entry object with the transparentengagement surface causes light from the illuminator to be scattered dueto frustrated total internal reflection.

Additionally in accordance with a preferred embodiment of the presentinvention the illuminator provides illumination generally through 360degrees and the two-dimensional imaging sensor views generally through360 degrees.

Preferably, the illuminator provides a non-uniform intensitydistribution.

Further in accordance with a preferred embodiment of the presentinvention at least a portion of the non-uniform intensity distributionprovides greater intensity at greater illumination angles.

Still further in accordance with a preferred embodiment of the presentinvention the data input device also includes a data entry object speedsensor operative to sense the speed with which the data entry objectapproaches the engagement plane.

Preferably, the illuminator includes at least first and secondwavelength specific illuminators operative at least first and seconddifferent wavelengths and directing light along at least first andsecond mutually spaces, overlying engagement planes and thetwo-dimensional imaging sensor senses light at the first and seconddifferent wavelengths, differentiates therebetween and provides anoutput to the data entry object speed sensor.

Further in accordance with a preferred embodiment of the presentinvention the illuminator includes at least first and secondilluminators operative at the same wavelength and directing light alongat least first and second mutually spaced, overlying engagement planesand the data entry object speed sensor is responsive to changes in theintensity of light senses by the two-dimensional imaging sensor forproviding an output indication of the speed.

Preferably, the illuminator directs light, which is emitted from a pointsource through a large solid angle, into a flat radially directed beamextending along the engagement plane, the beam having a relativelynarrow spread in a direction perpendicular to the engagement plane.

Still further in accordance with a preferred embodiment of the presentinvention the illuminator includes a point light source which emitslight through a generally semi-hemispherical volume centered about apropagation axis, an spheric reflector which reflects the light emittedby the point light source along a line lying in the engagement plane andextending perpendicular to the propagation axis, the aspheric reflectorreflecting light from different elevations so that the reflected lightpasses through the line at differing locations therealong and a twistedelongate mirror, arranged along the line which reflects the lightpassing through the line a various elevation angles as a planar flatbeam which lies in a plane, which plane extends through the line andtraverses a slit in the aspheric reflector.

Preferably, the aspherical reflector includes strips of a sphericalmirror whose centers are offset from each other along an axis lying inthe engagement plane and extending perpendicular to the propagationaxis.

Preferably, the two-dimensional imaging sensor includes anangle-compensated interference filter.

Further in accordance with a preferred embodiment of the presentinvention the angle-compensated interference filter includes a pluralityof thin films, each being of non-uniform thickness, formed onto a domeshaped transparent substrate having an axis of symmetry.

Preferably, the plurality of thin films have a thickness which isselected to vary such that the thickness of the plurality of thin filmstraversed by light beams impinging onto a given point located along theaxis of symmetry is generally identical irrespective of the angularrelationship between the light beam and the axis of symmetry.

Additionally in accordance with a preferred embodiment of the presentinvention the data input device also includes an imaging lens located atthe given point, which directs the light to the two-dimensional imagingsensor.

Typically, the dome shaped transparent substrate is configured such thatuniform evaporation of film material thereonto from a location spacedtherefrom produces the plurality of thin films each being of non-uniformthickness which is selected to vary such that the thickness of theplurality of thin films traversed by light beams impinging onto a givenpoint located along the axis of symmetry is generally identicalirrespective of the angular relationship between the light beam and theaxis of symmetry.

Further in accordance with a preferred embodiment of the presentinvention the data entry processor is operative to map locations on thetwo-dimensional image sensor to data entry functions.

Preferably, the data entry processor is operative to map received lightintensity at the locations on the two-dimensional image sensor to thedata entry functions.

Further in accordance with a preferred embodiment of the presentinvention the data entry processor includes the following functionality:as each pixel value is acquired, determining, using the pixelcoordinates, whether that pixel lies within a predefined keystrokeregion, acquiring pixel values for various pixel coordinates, adding orsubtracting each pixel value to or from a pixel total maintained foreach the keystroke region based on determining a pixel function of eachpixel and comparing the pixel total for each the keystroke region with acurrent key actuation threshold. If the pixel total exceeds the keyactuation threshold for a given keystroke region in a given frame and inthe previous frame the pixel total did not exceed the key actuationthreshold for that keystroke region, provide a key actuation output.Additionally or alternatively, if the pixel total does not exceed thekey actuation threshold for a given keystroke region in a given frameand in the previous frame the pixel total did exceed the key actuationthreshold for that keystroke region, provide a key deactuation output.

Preferably, the data input device determines whether that pixel lieswithin a predefined keystroke region is made by employing a pixel indextable which indicates for each pixel, whether that pixel lies within apredetermined keystroke region and, if so, within which keystroke regionit lies.

Further in accordance with a preferred embodiment of the presentinvention both determining steps employ the pixel index table.

Preferably, the pixel total is maintained for each keystroke region in akeystroke region accumulator table.

Preferably, the comparing step employs a keystroke region thresholdtable.

Still further in accordance with a preferred embodiment of the presentinvention the data input device also includes the followingfunctionality: once all of the pixels in a frame have been processed,determining an updated background level for a frame and determining akey actuation threshold for the keystroke region threshold table bysubtracting the updated background level from a predetermined thresholdlevel which is established for each keystroke region.

Further in accordance with a preferred embodiment of the presentinvention the pixel function includes adding the pixel values of aplurality of pixels in the keystroke region.

Additionally or alternatively, the pixel function includes adding thepixel values of the plurality of pixels in the keystroke region andsubtracting therefrom pixel values of a plurality of pixels in akeystroke region border outside the keystroke region.

Additionally or alternatively, the pixel function includes adding thepixel values of the plurality of pixels in the keystroke region,ignoring the pixel values of a plurality of pixels in a first keystrokeregion border outside the keystroke region and subtracting pixel valuesof a plurality of pixels in a second keystroke region border, outsidethe first keystroke region border.

Further in accordance with a preferred embodiment of the presentinvention the data entry processor is operative to determine the “centerof gravity” of pixel values of pixels in the two-dimensional imagesensor.

Still further in accordance with a preferred embodiment of the presentinvention the data entry processor includes the following functionality:as each pixel value is acquired, determining, using the pixelcoordinates, whether that pixel lies within a predefined active region,acquiring pixel values for various pixel coordinates and determining the“center of gravity” of the pixel values.

Preferably, the step of determining the “center of gravity” is achievedby: multiplying the pixel values by X and Y values representing thegeographic position of each pixel, summing the results along mutuallyperpendicular axes X and Y, summing the total of the pixel values forall relevant pixels for the active region and dividing the summedresults by the total of the pixel values to determine the X and Ycoordinates of the “center of gravity”, which represents a desiredengagement location.

Typically, the pixel values are thresholded prior to summing thereof.

Further in accordance with a preferred embodiment of the presentinvention the non-uniform intensity distribution varies over time.

Preferably, the two-dimensional sensor operates to view differentimaging fields at different times and wherein the operation of theilluminator is correlated with the operation of the two-dimensionalimage sensor, whereby the intensity of light produced by the illuminatorvaries in synchronization with an imaging field location of thetwo-dimensional image sensor.

Preferably, the distance between the two-dimensional sensor and its theimaging field location increases, the intensity of light provided by theilluminator increases.

Typically, the data input device also includes variable intensity driveelectronics which is coupled to the illuminator and to thetwo-dimensional detector and which causes the intensity of lightproduced by the illuminator to vary in synchronization to the imagingfield location of the two-dimensional detector.

Still further in accordance with a preferred embodiment of the presentinvention the data input device also includes a digital signaturegenerator receiving an input from the data entry processor includingintensity, position and timing outputs and employs the outputs toprovide a digital signature.

There is also provided in accordance with a preferred embodiment of thepresent invention a data input device, which includes an illuminatoroperative to illuminate at least one engagement surface, atwo-dimensional imaging sensor viewing the engagement surface from alocation outside the engagement surface for sensing engagement of a dataentry object with the engagement surface and a data entry processorreceiving an output from the two-dimensional imaging sensor andproviding a data entry input to utilization circuitry, the data entryprocessor employing shadow analysis.

There is further provided in accordance with a preferred embodiment ofthe present invention a data input method, which includes illuminatingat least one engagement surface, viewing the engagement surface with atwo-dimensional image sensor from a location outside the engagementsurface for sensing engagement of a data entry object with theengagement surface and processing an output from the two-dimensionalimaging sensor and providing a data entry input to utilizationcircuitry, the data entry processor employing shadow analysis.

Further in accordance with a preferred embodiment of the presentinvention the illuminator includes a non-point light source and the dataentry processor employs a shadow density analyzer to determine thesharpness of edges of a shadow defined by the non-point light source andthe data entry object on the engagement surface, which indicates thepropinquity of the data entry object to the projection surface.

Additionally or alternatively, the illuminator includes a plurality oflight sources and the data entry processor employs a shadow coalescenceanalyzer to determine the extent of coalescence of shadows defined bythe plurality of light sources and data entry object on the engagementsurface, which indicates the propinquity of the data entry object to theprojection surface.

Preferably, the data entry processor includes the followingfunctionality: as each pixel value is acquired, determining, using thepixel coordinates, whether that pixel lies within a predefined keystrokeregion and within predefined left and right keystroke subregionstherewithin, acquiring pixel values for various pixel coordinates,obtaining the derivative of each pixel value along an X axis, summingthe derivatives for each the subregion, one from the other to provide adifference and comparing the difference with a current key actuationthreshold. If the difference exceeds the key actuation threshold for agiven keystroke region in a given frame and in the previous frame thepixel total did not exceed the key actuation threshold for thatkeystroke region, provide a key actuation output. Additionally oralternatively, if the difference does not exceed the key actuationthreshold for a given keystroke region in a given frame and in theprevious frame the pixel total did exceed the key actuation thresholdfor that keystroke region, provide a key deactuation output.

Preferably, the step of determining employs a pixel index table, whichindicates for each pixel, whether that pixel lies within a predeterminedkeystroke region and, if so, within which keystroke region as well aswithin which keystroke subregion it lies.

Typically, the pixel total is maintained for each keystroke subregion ina keystroke subregion accumulator table.

Still further in accordance with a preferred embodiment of the presentinvention the step of comparing employs a keystroke region thresholdtable.

Additionally in accordance with a preferred embodiment of the presentinvention the engagement plane is associated with a pull-down tray in avehicle wherein the pull-down tray defines an engagement surface whichis configured by projection.

Further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator are associatedwith a camera.

Still further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator are associatedwith a home entertainment system.

Additionally in accordance with a preferred embodiment of the presentinvention the engagement plane overlies a television screen forming partof the home entertainment system.

Further in accordance with a preferred embodiment of the presentinvention the engagement plane is associated with a table.

Still further in accordance with a preferred embodiment of the presentinvention the engagement plane is associated with a remote controldevice.

Additionally in accordance with a preferred embodiment of the presentinvention the engagement plane is located within a restrictedparticulate manner environment.

Further in accordance with a preferred embodiment of the presentinvention the engagement plane is located within an industrialenvironment unsuitable for a conventional keyboard.

Preferably, the two-dimensional detector and illuminator are associatedwith a video projector.

Still further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator are associatedwith a restaurant patron interface system.

Additionally in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator are associatedwith a mobile audio player.

Further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator provide touchscreen functionality.

Preferably, the touch screen functionality employs a video displayscreen.

Still further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator provide accesscontrol functionality.

Preferably, the engagement plane is associated with a game board andwherein the game board defines an engagement surface, which isconfigured by projection.

Additionally in accordance with a preferred embodiment of the presentinvention the engagement plane is associated with a musical instrumentand wherein the musical instrument defines an engagement surface, whichis configured by projection.

Further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator provide vehicletelematics functionality. Preferably, the vehicle defines an engagementsurface, which is configured by projection.

Still further in accordance with a preferred embodiment of the presentinvention the two-dimensional detector and illuminator provide automaticvending user interface functionality.

There is further provided in accordance with another preferredembodiment of the present invention an angle-compensated interferencefilter which includes a plurality of thin films, each being ofnon-uniform thickness, formed onto a dome shaped transparent substratehaving an axis of symmetry. The plurality of thin films have athickness, which is selected to vary, such that the thickness of theplurality of thin films traversed by light beams impinging onto a givenpoint located along the axis of symmetry is generally identicalirrespective of the angular relationship between the light beam and theaxis of symmetry.

There is also provided in accordance with a further preferred embodimentof the present invention a method for filtering light employing anangle-compensated interference filter, which includes a plurality ofthin films, each being of non-uniform thickness, formed onto a domeshaped transparent substrate having an axis of symmetry. The pluralityof thin films have a thickness which is selected to vary such that thethickness of the plurality of thin films traversed by light beamsimpinging onto a given point located along the axis of symmetry isgenerally identical irrespective of the angular relationship between thelight beam and the axis of symmetry.

Further in accordance with a preferred embodiment of the presentinvention the dome shaped transparent substrate is configured such thatevaporation of film material thereonto from a location spaced therefromproduces the plurality of thin films each being of non-uniformthickness. The non-uniform thickness is selected to vary such that thethickness of the plurality of thin films traversed by light beamsimpinging onto a given point located along the axis of symmetry isgenerally identical irrespective of the angular relationship between thelight beam and the axis of symmetry.

Preferably, the step of evaporation is performed in a uniform matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully fromthe following detailed description, taken in conjunction with thedrawings in which:

FIG. 1 is a simplified and generalized illustration of a projectionkeyboard system and methodology, constructed and operative in accordancewith a preferred embodiment of the present invention;

FIG. 2 is a simplified illustration of a keyboard projection subsystememploying a diffractive optical element and having optical power,constructed and operative in accordance with a preferred embodiment ofthe present invention;

FIGS. 3A and 3B are respective simplified pictorial and top viewillustrations of a keyboard projection subsystem employing an integrallyformed beam splitter and diffractive optical elements, constructed andoperative in accordance with a preferred embodiment of the presentinvention;

FIG. 4 is a simplified illustration of a multiple format keyboardprojection subsystem employing a plurality of different diffractiveoptical elements which are selectably positionable along a keyboardprojection light path, constructed and operative in accordance with apreferred embodiment of the present invention;

FIGS. 5A and 5B are respective simplified pictorial and side viewillustrations of a keyboard projection subsystem employing a diffractiveoptical element having diffraction orders selected to provide a keyboardconfiguration which has a relatively low maximum diffraction angle,constructed and operative in accordance with a preferred embodiment ofthe present invention;

FIGS. 6A and 6B are respective simplified pictorial and top viewillustrations of a keyboard projection subsystem employing a beamcombiner, constructed and operative in accordance with a preferredembodiment of the present invention;

FIG. 7 is a simplified illustration of a keyboard projection subsystememploying an array of light emitting elements and microlenses,constructed and operative in accordance with a preferred embodiment ofthe present invention;

FIG. 8 is a simplified illustration of a keyboard projection subsystememploying specially configured light emitting elements, constructed andoperative in accordance with a preferred embodiment of the presentinvention;

FIGS. 9A and 9B are respective pictorial and side view illustrations ofa data entry object engagement location sensing subsystem employing acamera located on the opposite side of a transparent data entry objectengagement surface from a data entry object engagement location sensingilluminator, constructed and operative in accordance with a preferredembodiment of the present invention;

FIGS. 10A and 10B are respective pictorial and side view simplifiedillustrations of a data entry object engagement location sensingsubsystem employing a transparent data entry object engagement surfaceexhibiting total internal reflection, constructed and operative inaccordance with a preferred embodiment of the present invention;

FIGS. 11A and 11B are simplified illustrations of a data entry objectengagement location sensing subsystem employing shadow sharpnessanalysis, constructed and operative in accordance with a preferredembodiment of the present invention;

FIGS. 12A and 12B are simplified illustrations of a data entry objectengagement locations sensing subsystem employing shadow coalescencesensing, constructed and operative in accordance with a preferredembodiment of the present invention;

FIGS. 13A and 13B are simplified illustrations of a data entry objectengagement location sensing subsystem having a 360 degree detectionrange, constructed and operative in accordance with a preferredembodiment of the present invention;

FIG. 14A is a simplified illustration of an illumination subsystemincluding an illuminator which provides desired non-uniform illuminationintensity and employing an aspheric element, constructed and operativein accordance with a preferred embodiment of the present invention;

FIG. 14B is a simplified illustration of an illumination subsystemincluding an illuminator which provides desired non-uniform illuminationintensity and employing a diffractive element, constructed and operativein accordance with a preferred embodiment of the present invention;

FIG. 14C is a simplified illustration of an illumination subsystemincluding an illuminator which provides desired non-uniform illuminationintensity and employing a combination of cylindrical lenses, constructedand operative in accordance with a preferred embodiment of the presentinvention;

FIGS. 15A and 15B are respective simplified pictorial and side viewillustrations of a data entry object engagement location sensingsubsystem including a data entry object engagement speed sensor havingplural illumination and detection planes and employing pluralilluminators and sensors, constructed and operative in accordance with apreferred embodiment of the present invention;

FIGS. 16A and 16B are respective simplified pictorial and sectionalillustrations of a data entry object engagement location sensingsubsystem including a data entry object engagement speed sensor havingplural illumination and detection planes and employing pluralilluminators and sensors, constructed and operative in accordance with apreferred embodiment of the present invention;

FIGS. 17A and 17B are respective simplified pictorial and sectionalillustrations of a data entry object engagement location sensingsubsystem including a data entry object engagement speed sensor havingplural illumination and detection planes and employing a singleilluminator and a single sensor, constructed and operative in accordancewith a preferred embodiment of the present invention;

FIG. 18 is a simplified illustration of an illuminator useful in a dataentry object engagement location sensing subsystem and employingaspheric reflectors, constructed and operative in accordance with apreferred embodiment of the present invention;

FIG. 19 is a simplified illustration of a angle-compensated interferencefilter employed in data entry object engagement location sensingsubsystem, constructed and operative in accordance with a preferredembodiment of the present invention;

FIG. 20 is a simplified flow chart illustrating operation of a dataentry object engagement location sensing subsystem employed in theprojection keyboard system and methodology of FIG. 1 in accordance witha preferred embodiment of the present invention;

FIG. 21 is a simplified illustration of a preferred data structureemployed in the operation of the data entry object engagement locationsensing subsystem shown in FIG. 20;

FIG. 22 is a simplified pictorial illustration of outlines of typicalkeystroke regions as senses by a two-dimensional image sensor viewing akeyboard, such as the keyboard seen in FIG. 5A;

FIG. 23 is a simplified pictorial illustration of outlines of typicalfootprints of a typical light pattern occasioned by data entry objectengagement with several keystroke regions, such as those shown in FIG.22;

FIGS. 24A, 24B and 24C are simplified illustrations of three alternativemethodologies for determining the function of the pixel within thekeystroke region in which it lies as shown in FIG. 21;

FIGS. 25A, 25B and 25C are simplified illustrations of traces which areuseful in understanding FIGS. 24A, 24B and 24C;

FIG. 26 is a simplified flow chart illustrating operation of a dataentry object engagement location sensing subsystem employed in atracking system and methodology constructed and operative in accordancewith a preferred embodiment of the present invention;

FIG. 27 is a simplified flowchart illustrating operation offunctionality providing shadow sharpness analysis in accordance with apreferred embodiment of the present invention;

FIG. 28 is a simplified illustration of a preferred data structureemployed in the operation of the data entry object engagement locationsensing subsystem shown in FIG. 27;

FIG. 29 is an illustration which is useful in understanding theflowchart of FIG. 27;

FIG. 30 is a simplified illustration showing synchronized illuminationpower variation functionality useful in accordance with a preferredembodiment of the present invention;

FIG. 31 is a simplified illustration of a system and functionality forproviding a digital signature in accordance with a preferred embodimentof the present invention;

FIG. 32 is a simplified illustration of a keyboard system andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention and employing sensing of a dataentry object interaction with an inert keyboard defined on a pull-downtray;

FIG. 33 is a simplified illustration of a keyboard system andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention and providing alphanumericannotation of photographs using a suitably equipped camera;

FIGS. 34A, 34B, 34C and 34D are simplified illustrations of fouralternative embodiments of a keyboard system and methodology,constructed and operative in accordance with a preferred embodiment ofthe present invention and providing control, by data entry objectinteraction, of a home entertainment system;

FIG. 35 is a simplified illustration of a restricted particulate matterenvironment keyboard system and methodology, constructed and operativein accordance with a preferred embodiment of the present invention;

FIG. 36 is a simplified illustration of a industrial environmentkeyboard system and methodology, constructed and operative in accordancewith a preferred embodiment of the present invention;

FIG. 37 is a simplified illustration of a video projector havingintegrally formed or associated therewith a keyboard system andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention;

FIG. 38 is a simplified illustration of a restaurant patron interfacesystem and methodology, constructed and operative in accordance with apreferred embodiment of the present invention;

FIG. 39 is a simplified illustration of a keyboard system andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention;

FIGS. 40A and 40B are simplified illustrations of a data entry objectengagement sensing screen system and methodology, constructed andoperative in accordance with a preferred embodiment of the presentinvention;

FIG. 41 is a simplified illustration of a security and access controlsystem employing data entry object engagement sensing methodology,constructed and operative in accordance with a preferred embodiment ofthe present invention;

FIG. 42 is a simplified illustration of a object engagement sensing gamesystem and methodology, constructed and operative in accordance with apreferred embodiment of the present invention;

FIGS. 43A, 43B and 43C are simplified illustrations of a data entryobject engagement sensing musical instrument and methodology,constructed and operative in accordance with a preferred embodiment ofthe present invention;

FIG. 44 is a simplified illustration of a vehicle mounted user interfacesystem and methodology, constructed and operative in accordance with apreferred embodiment of the present invention; and

FIG. 45 is a simplified illustration of a vending machine incorporatinga data entry object engagement detection system and methodology,constructed and operative in accordance with a preferred embodiment ofthe present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference is now made in FIG. 1, which is a simplified and generalizedillustration of a projection keyboard system and methodology,constructed and operative in accordance with a preferred embodiment ofthe present invention. A projection subsystem 100, preferably includinga solid state light source (not shown) which illuminates a spatial lightmodulation element (not shown), defines an image 102 of a keyboard on aprojection surface 104, preferably an inert surface, such as a desktop.

An illumination subsystem 106, preferably including a solid state lightsource (not shown), direct light in a radially directed illuminationpattern 108, which preferably extends in a plane generally parallel tothe projection-surface 104. It is appreciated that the radially directedillumination pattern 108 has a very narrow spread in the directiongenerally perpendicular to the projection surface 104. It is furtherappreciated that the radially directed illumination pattern 108 ispreferably located very close to the projection surface 104.

Impingement of light in the radially directed illumination pattern 108on a data entry object 110, such as a user's finger, a stylus or otheruse implement, causes light to be scattered or reflected therefrom. Itis appreciated that the light is only scattered or reflected when thedata entry object 110 is in close contact with the keyboard 102 definedon projections surface 104.

A detection subsystem 112, preferably employing a solid state imagingsensor (not shown), receives light reflected or scattered from dataentry object 110. The received light is mapped onto the imaging sensorwith respect to a representation of the keyboard for associating thelocation of the data entry object 110 sensed by detection subsystem 112with a key location 113 on the keyboard 102. Thus the sensed location ofdata entry object 110 indicates which key of the keyboard 102 is beingengaged.

Reference is now made to FIG. 2, which is a simplified illustration of apreferred embodiment of a keyboard projection subsystem 100 (FIG. 1)employing a diffractive optical element 120 which receives light from alight source 122, such as a diode laser, via a collimating lens 124.Light passing through the diffractive optical element 120 preferably isreflected by a curved mirror 126 having optical power, optionally via alens 128 onto projection surface 104 (FIG. 1), preferably defining akeyboard 102 (FIG. 1).

In accordance with a preferred embodiment of the present invention, thediffractive optical element 120, the mirror 126 and optionally the lens128 are all integrally formed in a suitably configured prism 130, as byembossing.

The configuration of FIG. 2 is preferred in order to enable adiffractive optical element having a relatively small maximumdiffraction angle to be used. The mirror 126, having optical powerspreads the light passing through the diffractive optical element 120 toa sufficient degree to enable projection of a full sized keyboard 102 onprojection surface 104, even when projection surface 104 is relativelyclose to the diffractive optical element 120. Alternatively, the prism103 and the mirror 126 may be replaced by one or more lenses.

Reference is now made to FIGS. 3A and 3B, which are respectivesimplified pictorial and top view illustrations of the keyboardprojection subsystem 100 (FIG. 1) employing an integrally formed beamsplitter and diffractive optical elements, constructed and operative inaccordance with a preferred embodiment. This embodiment employs a beamsplitter 140 preferably integrally formed with plural diffractiveoptical elements 142. A beam of light from a light source 144, such as adiode laser, preferably passes through a collimating lens 146 andimpinges on two mutually angles surfaces 148 of beam splitter 140. Thebeam splitter 140 breaks the beam of light into two beams, each of whichpasses through a separate diffractive optical element 142. Light passingthrough both diffractive optical elements 142 impinges on projectionsurface 104 (FIG. 1), preferably defining a keyboard 102 (FIG. 1).

In accordance with a preferred embodiment of the present invention, thediffractive optical elements 142 are integrally formed with the beamsplitter 140 in a suitably configured prism, as by embossing.

The configuration of FIGS. 3A and 3B is preferred in order to enable apair of diffractive optical elements, each having a relatively smallmaximum diffraction angle, to be used in combination together to definea full sized keyboard 102 on projection surface 104, even whenprojection surface 104 is relatively close to the diffractive opticalelement 120. An added advantage of using plural diffractive opticalelements is an increase in resolution, inasmuch as each diffractiveoptical element defines only part of the image projected onto projectionsurface 104. Preferably, the beam splitter 140 is configured such thatthe tow beams each impinge perpendicularly onto a correspondingdiffractive optical element 142.

Reference is now made to FIG. 4, which is a simplified illustration of apreferred multiple format embodiment of keyboard projection subsystem100 (FIG. 1). This embodiment employs a plurality of differentdiffractive optical elements 160, each of which typically corresponds toa different keyboard configuration. The optical elements 160 arepreferably mounted onto a rotatable support 162 in order to beselectably positionable along a keyboard projection light path 164extending from a light source 166, such as a diode laser, preferablythrough a collimating lens 168 and preferably impinging on a mirror 170,which directs light passing therealong onto projection surface 104 (FIG.1), preferably defining a keyboard 102 (FIG. 1).

Reference is now made to FIGS. 5A and 5B, which are simplifiedillustrations of a keyboard projection subsystem employing a diffractiveoptical element 180 having a multiplicity of diffraction orders 182selected to provide a keyboard configuration which has a relatively lowmaximum diffraction angle 184. Angle 184 is preferably in excess of 90degrees and is typically between 60 degrees and 120 degrees.

As seen in FIG. 5A, light from a light source 186 passing through acollimating lens 188 and thence through the diffractive optical element180 preferably falls onto a projection surface 104 (FIG. 1), preferablydefining a generally trapezoidal shaped keyboard 190, which isconfigured in accordance with a preferred embodiment of the presentinvention.

The diffraction orders 182 of the diffractive optical element 180 arecalculated and selected to compensate for geometrical distortionsinherent in the operation of a diffractive optical element, such aselement 180, particularly at high diffraction angles, such as angle 184.To this end, the individual diffraction orders 182 are preferablyarranged in rows 194 which extend obliquely with respect to lines 196defined thereby.

Additionally, the diffraction orders 182 are calculated and selected inorder to compensate for geometric distortions occasioned by a highlyoblique angle of projection, such as angle 192, seen in FIG. 5B. To thisend the diffraction orders are arranged as shown in FIG. 5A, to have abarrel-like distortion and to have a non-uniform outwardly increasingspacing between lines which are sought to appear parallel on keyboard190. Angle 192 is preferably less than less than 30 degrees and istypically between 20 degrees and 90 degrees.

Reference is now made to FIGS. 6A and 6B, which are simplifiedillustrations of a keyboard projection subsystem employing a beamcombiner 200. As seen in FIGS. 6A and 6B, light from a pair of pointlight sources 202 and 204 is combined by beam combiner 200, such thattwo light beams 206 and 208 emerge from the beam combiner 200 and appearto originate in a single virtual light source 210 positioned behind beamcombiner 200. In actuality the two light beams 206 and 208 nearlyoverlap, but may define a no-light beam region 212 therebetween.

The light beams 206 and 208 pass through a shadow mask 214 ontoprojection surface 104 (FIG. 1), preferably defining a keyboard 102(FIG. 1).

The embodiment of FIGS. 6A and 6B has an advantage in that it may employmultiple relatively low power and low cost laser diodes to provide thesame power as would be provided by a single much more expensive laserdiode.

Reference is now made to FIG. 7, which is a simplified illustration of akeyboard projection subsystem employing an array 230 of light emittingelements 232 and microlenses 234. As seen in FIG. 7, light from multiplepoint light emitting elements 232, such as LEDs, is imaged bycorresponding multiple microlenses 234 onto projection surface 104 (FIG.1), preferably defining a portion of keyboard 102 (FIG. 1), such as theletter “E”. It is appreciated that each of light emitting elements 232is individually controllable in order to provide a correspondinglyindividual light spot 236 on projection surface 104. The collection oflight spots 236 makes up the keyboard 102 (FIG. 1). The embodiment ofFIG. 7 provides a selectable and changeable keyboard.

Reference is now made to FIG. 8, which is a simplified illustration of akeyboard projection subsystem employing specially configured lightemitting elements, preferably a monolithic pattern 250 of LEDs formed ona unitary substrate 252.

As seen in FIG. 8, light from the pattern 250 is imaged by a lens 254onto projection surface 104 (FIG. 1), preferably defining keyboard 102(FIG. 1). This arrangement has the advantage of electrical efficiencyand low unit cost but does not provide a variable keyboardconfiguration.

Reference is now made to FIGS. 9A and 9B, which are respective pictorialand side view illustrations of a data entry object engagement locationsensing subsystem employing a camera 270 located on the opposite side ofa transparent data entry object engagement surface 272 from a data entryobject engagement location sensing illuminator 274. A generally flatplanar beam of light, designated by reference numeral 276, is preferablyemitted by illuminator 274 generally parallel to and spaced from dataentry object engagement surface 272. As seen particularly in FIG. 9B,the presence of an object, such as a data entry object 278 in beam 276,causes light from beam 276 to be scattered into a scattered beam 280 andinter alia to pass through transparent data entry object engagementsurface 272 so as to be detected by camera 270, which preferably formspart of detection subsystem 112 (FIG. 1).

Reference is now made to FIGS. 10A and 10B, which are respectivepictorial and side view simplified illustrations of a data entry objectengagement location sensing subsystem employing a transparent data entryobject engagement surface 290, exhibiting total internal reflection. Aplanar beam of light, designated by reference numeral 292, is emitted byan illuminator 294 and coupled to an edge 295 of surface 290 throughwhich beam 292 passes by total internal reflection. As seen particularlyin FIG. 10B, the presence of an object, such as a data entry object 296in contact with surface 290, causes light from bean 292 to be scatteredinto a scattered beam 297 due to frustrated total internal reflectionand inter alia to pass through transparent data entry object engagementsurface 290 so as to be detected by a camera 298, which preferably formspart of detection subsystem 112 (FIG. 1).

Reference is now made to FIGS. 11A and 11B, which are a simplifiedillustration of a data entry object engagement location sensingsubsystem, forming part of detection subsystem 112 (FIG. 1) andemploying shadow sharpness analysis, constructed and operative inaccordance with a preferred embodiment of the present invention. Anobject, such as a data entry object 300, casts a shadow 302 on aprojection surface 104 (FIG. 1) when illuminated by a light source 304.A camera 306 senses the shadow and a shadow density analyzer 308,determines the optical density of the shadow, which indicates thepropinquity of the data entry object 300 to projection surface 104.

Reference is now made to FIGS. 12A and 12B, which are simplifiedillustrations of a data entry object engagement location sensingsubsystem forming part of detection subsystem 112 (FIG. 1) and employingshadow coalescence sensing. An object, such as a data entry object 320,casts shadows 322 and 324 on a projection surface 104 (FIG. 1) whenilluminated by a pair of infrared point light sources 326 and 328, suchas LEDs. A camera 330 senses the shadows 322 and 324 and a shadowcoalescence sensor 332 determines the extent of overlap or theseparation between the shadows 322 and 324, which indicates thepropinquity of the data entry object 320 to projection surface 104.

Reference is now made to FIGS. 13A and 13B, which are simplifiedillustrations of a data entry object engagement location sensingsubsystem 340 having a 360 degree annular detection range 342. The dataentry object engagement location sensing subsystem 340 of FIG. 13preferably includes an illuminator 344, such as a diode laser, providingan generally conical output beam 346 which impinges on a generallyconical mirror 348, which provides via an annular window 350, agenerally planar, radially directed illumination beam 351, generallyparallel to the projection surface 104 (FIG. 1), such as a table top352. A camera 354 views a generally annular range 342 defined betweenvirtual circles 356 and 358 on table top 352 and senses light scatteredby objects, such as data entry object tips 360. Preferably, thescattered light is received by camera 354 via a conical mirror 362 andvia an annular window 364.

Reference is now made to FIG. 14A, which is a simplified illustration ofan illumination subsystem 106 (FIG. 1) including an illuminator 370,preferably including a diode laser light source 372, a collimating lens374 and an aspheric element 376, such as an aspheric cylindrical lens,receiving light from the light source 372 via the collimating lens 374.The aspheric element 376 preferably directs light in a radially directedillumination pattern 378, which preferably extends in a plane generallyparallel to the projection surface 104 (FIG. 1). It is appreciated thatthe radially directed illumination pattern 378 has a very narrow spreadin the direction generally perpendicular to the projection surface 104.It is further appreciated that the radially directed illuminationpattern 378 is preferably located very close to the projection surface104.

The illumination subsystem of FIG. 14A provides the desired spatiallynon-uniform illumination intensity pattern 378, wherein the intensityvaries as a function of the illumination angle 379, as seen for example,at graph 380. It is noted that greater illumination intensity isprovided at large illumination angles in order to compensate for thenon-uniform detection effects at the large viewing angles. Thesenon-uniform detection effects include the reduction of the effectiveangular cross-section of the data entry object 110 (FIG. 1) and thereduced light collection efficiency of the lens on the camera in thedetection subsystem 112 (FIG. 1).

Reference is now made to FIG. 14B, which is a simplified illustration ofthe illumination subsystem 106 (FIG. 1) including an illuminator 390,preferably including a diode laser light source 392, a collimating lens394 and an diffractive optical element 396, receiving light from thelight source 392 via the collimating lens 394. The diffractive opticalelement 396 preferably directs light in a radially directed illuminationpattern 398, which preferably extends in a plane generally parallel tothe projection surface 104 (FIG. 1). It is appreciated that the radiallydirected illumination pattern 398 has a very narrow spread in thedirection generally perpendicular to the projection surface 104. It isfurther appreciated that the radially directed illumination pattern 398is preferably located very close to the projection surface 104.

The illumination subsystem of FIG. 14B provides the desired spatiallynon-uniform illumination intensity pattern 398, wherein the intensityvaries as a function of the illumination angle 399, as seen for example,at graph 400. It is noted that greater illumination intensity isprovided at large illumination angles in order to compensate for thenon-uniform detection effects at the large viewing angles. Thesenon-uniform detection effects include the reduction of the effectiveangular cross-section of the data entry object 110 (FIG. 1) and thereduced light collection efficiency of the lens on the camera in thedetection subsystem 112 (FIG. 1).

Reference is now made to FIG. 14C, which is a simplified illustration ofthe illumination subsystem 106 (FIG. 1) including an illuminator 410,preferably including a diode laser light source 412, a collimating lens414 and a joined double side-truncated rod lens optical element 416,receiving light from the light source 412 via the collimated lens 414.The optical element 416 preferably directs light in a radially directedillumination pattern 418, which preferably extends in a plane generallyparallel to the projection surface 104 (FIG. 1). It is appreciated thatthe radially directed illumination patter 418 has a very narrow spreadin the direction generally perpendicular to the projection surface 104.It is further appreciated that the radially directed illuminationpattern 418 is preferably located very close to the projection surface104.

The illumination subsystem of FIG. 14C provides the desired spatiallynon-uniform illumination intensity pattern 418, wherein the intensityvaries as a function of the illumination angle 419, as seen for example,at graph 420. It is noted that greater illumination intensity isprovided at large illumination angles in order to compensate for thenon-uniform detection effects at the large viewing angles. Thesenon-uniform detection effects include the reduction of the effectiveangular cross-section of the data entry object 110 (FIG. 1) and thereduced light collection efficiency of the lens on the camera in thedetection subsystem 112 (FIG. 1).

The precise illumination distribution may be selected by suitablevariation of the radii R of the side-truncated rod lenses 422 and 424and the extent X of their mutual side truncation.

Reference is now made to FIGS. 15A and 15B, which are respectivesimplified pictorial and sectional illustrations of a data entry objectengagement location sensing subsystem including a data entry objectengagement speed sensor having plural illumination and detection planesand employing plural illuminators and sensors, constructed and operativein accordance with a preferred embodiment of the present invention.

As seen in FIGS. 15A and 15B, first and second generally flat mutuallyspaced and overlying planar beams of light of differing wavelengths,designated respectively by reference numeral 430 and 432, are preferablyemitted by respective illuminators 434 and 436 generally parallel to andspaced from a data entry object engagement surface 438. As seenparticularly in FIG. 15B, the presence of an object, such as a dataentry object 440 in beams 430 and 432, causes light from the respectivebeams to be scattered into scattered beams 439 and 441 and to bedetected by respective cameras 442 and 444, which have detectionwavelengths corresponding to those of beams 430 and 432 respectively.The cameras may be equipped with suitable filters 446 and 448 for thispurpose. Illuminators 434 and 436 form part of illumination subsystem106 (FIG. 1) while cameras 442 and 444 form part of detection subsystem112 (FIG. 1).

The data entry object engagement location sensing subsystem of FIGS. 15Aand 15B also includes a timing analyzer 450, which receives outputs fromcameras 442 and 444 and determines from the timing thereof, the speed ofengagement of the data entry object with data entry object engagementsurface 438. The speed of engagement of the data entry object with dataentry object engagement surface 438 may be employed in variousapplications, such as musical instruments and games.

Reference is now made to FIGS. 16A and 16B, which are respectivesimplified pictorial and sectional illustrations of a data entry objectengagement location sensing subsystem including a data entry objectengagement speed sensor having plural illumination and detection planesand employing plural illuminators and a single sensor, constructed andoperative in accordance with a preferred embodiment of the presentinvention.

As seen in FIGS. 16A and 16B, first and second generally flat planarmutually spaced and overlying beams of light of differing wavelengths,designated respectively by reference numeral 460 and 462, are preferablyemitted by respective illuminators 464 and 466 generally parallel to andspaced from a data entry object engagement surface 468. As seenparticularly in FIG. 16B, the presence of an object, such as a dataentry object 470 in beams 460 and 462, causes light from the respectivebeams to be scattered and to be detected by a camera 472, having firstand second detection regions 474 and 476, which have detectionwavelengths corresponding to those of beams 460 and 462 respectively.The detection regions of camera 472 are preferably defined by suitablefilters to provide desired wavelength differentiation. Illuminators 464and 466 form part of illumination subsystem 106 (FIG. 1) while, camera472 forms part of detection subsystem 112 (FIG. 1).

Light scattered by data entry object 470 from beams 460 and 462 ispreferably refracted by a prism 478 and split into two beams 480 and 482which are imaged by a lens 484 onto the two detection regions 474 and476.

The data entry object engagement location sensing subsystem of FIGS. 16Aand 16B also includes a timing analyzer 486, which receives outputs fromcamera 472 and determines from the timing thereof, the speed ofengagement of the data entry object 470 with data entry objectengagement surface 468. The speed of engagement of the data entry objectwith data entry object engagement surface 468 may be employed in variousapplications, such as musical instruments and games.

Reference is now made to FIGS. 17A and 17B, which are respectivesimplified pictorial and sectional illustrations of a data entry objectengagement location sensing subsystem including a data entry objectengagement speed sensor having plural illumination and detection planesand employing a single illuminator and a single sensor, constructed andoperative in accordance with a preferred embodiment of the presentinvention.

As seen in FIGS. 17A and 17B, first and second generally flat mutuallyspaced and overlying planar beams of light, designated respectively byreference numeral 500 and 502, are preferably emitted by an illuminator504 which outputs via a beam splitter 506 and a mirror 508. Beams 500and 502 are generally parallel to and spaced from a data entry objectengagement surface 510. As seen particularly in FIG. 17B, the presenceof an object, such as a data entry object 512 in beams 500 and 502,causes light from the respective beams to be scattered and to be imagedby a lens 514 into a camera 516. Illuminator 504 forms part ofillumination subsystem 106 (FIG. 1) while camera 516 forms part ofdetection subsystem 112 (FIG. 1).

The data entry object engagement location sensing subsystem of FIGS. 17Aand 17B also includes an intensity timing analyzer 518, which receivesan output from cameras 516 and determines from the timing of a stepwiseincrease in detected light intensity thereat, the speed of engagement ofthe data entry object with data entry object engagement surface 510. Thespeed of engagement of the data entry object with data entry objectengagement surface 510 may be employed in various applications, such asmusical instruments and games.

Reference is now made to FIG. 18, which is a simplified illustration ofan illuminator useful in a data entry object engagement location sensingsubsystem and employing aspheric reflectors, constructed and operativein accordance with a preferred embodiment of the present invention. Itis appreciated that the illuminator of FIG. 18 of the present inventiondirects light, which is emitted from a point source through a largesolid angle, into a flat radially directed beam extending along anengagement plane. The beam has a very narrow spread in a directionperpendicular to the projection surface 104 (FIG. 1).

As seen in FIG. 18, a point light source 550, such as an LED, emitslight through a generally semi-hemispherical volume denoted by referencenumeral 552. An aspheric reflector, strips of which are designated byreference numerals 554, 556 and 558, reflects the light emitted by thepoint light source 550 along a line 560, which typically passes throughthe light source 550. In a preferred embodiment of the presentinvention, the aspherical reflector may be constructed from strips of aspherical mirror whose centers have been offset from each other alongthe line 560. The aspheric reflector thus reflects light from differentelevations so that the reflected light passes through line 560 atdiffering locations therealong.

A twisted elongate mirror 562, preferably arranged along line 560,reflects the light passing through line 560 at various elevation anglesas a planar flat beam, denoted by reference numeral 564. Beam 564typically lies in a plane, which extends through line 560 and traversesa slit, not shown, appropriately positioned in the aspheric reflector.

Reference is now mad to FIG. 19, which is a simplified illustration ofan angle-compensated interference filter employed in a data entry objectengagement location sensing subsystem, constructed and operative inaccordance with a preferred embodiment of the present invention. Thefilter of FIG. 19 is useful in the present invention, for example asfilter 446 and 448 in FIGS. 15A and 15B and filters 474 and 476 in FIGS.16A and 16B.

As seen in FIG. 19, in an exaggerated form which is not drawn to scalefor the purposes of clarity, a plurality of thin films, collectivelydesignated by reference numeral 580, each being of non-uniform thicknessare formed onto a dome shaped curved transparent substrate 582, whichneed not be spherical, to define an interference filter. The thicknessof the thin films 580 is selected to vary over the substrate 582 suchthat the thickness of the thin films 580 traversed by every light beamimpinging onto a given point 584 located along an axis of symmetry 586of substrate 582 is identical irrespective of the angular relationshipbetween the light beam and the axis of symmetry 586 (OA in FIG. 19). Theimaging lens of a camera, such as camera 516 (FIG. 17A), is located atpoint 584.

Therefore, the intensity of the light beam reaching the camera 516 isindependent of the location of the keystroke, which is being engaged bydata entry object 512.

A preferred technique for the construction of the interference filter ofFIG. 19, by using methods, such as film evaporation, is set forthhereinbelow with reference to FIG. 19.

According to Snell's Law:Sin(α)=n·Sin(α¹)  (1)where α is the local incidence angle at the filter's surface of a raythat will eventually reach point O, α¹ is the local effective refractionangle at the filter's surface and n is the effective refractive index ofthe filter coating

Typically, in a first approximation, the optical axis of the dome shapedsubstrate 582 is in the direction of the evaporated material, which ispreferably used for manufacturing the interference filter. Additionally,in a first approximation, the flow of evaporated material on the dome isin a direction, which is typically perpendicular to the small region ofthe dome to which the material is being applied.

Thus, from mass conservation of the coating process, the thickness ofthe filter material in a direction θ, is given by:t(θ)=t·cos(θ)  (2)Thus, the length of a refracted ray, through the filter 582, is givenby:d(θ)=t(74)/Cos(α¹),where θ is the deflection angle between the normal to the filter at thepoint of incidence and the axis of symmetry 586 and t(θ) is the localthickness of the filter.

If the thickness of the filter is equal in all directions(eqi-filtering), thend(θ)=tandCos(θ)=Cos(α¹) or θ=α¹  (3)where d(θ) is the local path distance in the filter along the refractedray.

Therefore, equation (1) becomes:Sin(α)=n·Sin (θ)  (4)Using know trignometrical relationships, equation (1) may be written as:Cos (α)=√{square root over (1−π²·Sin²(θ))}  (5)As is known in the art, there are typically an infinite number ofsolutions, to equation (5), for the geometry of the dome 582.Preferably, one solution may be for the case for a typical light rayhitting the dome at angle α and defining a certain point P. The distancealong the optical axis from point P to the dome is given by R(θ).

According to the Sine Rule:

$\begin{matrix}{{\frac{{R(\theta)} - X}{R(\theta)} = \frac{{Sin}\;(\alpha)}{{Sin}\;\left( {\alpha + \theta} \right)}}{and}} & (6) \\{\frac{r(\theta)}{{Sin}\;(\theta)} = \frac{R(\theta)}{{Sin}\;\left( {\alpha + \theta} \right)}} & (7)\end{matrix}$where

-   R(θ) is the distance along the local normal to the filter between    the filter and point P;-   φ(θ) is the local deflection angle, such that φ=α+θ;-   X is a distance between the point 584 and the filter 582 in the    direction of OA;-   r(θ) is the distance between point 584 and the local incidence point    on the filter 582;

After substituting: equations (4) and (5) into equations (6) and (7),the following relationship may be obtained:

$\begin{matrix}{{\frac{{R(\theta)} - X}{R(\theta)} = {\frac{1}{{{Cos}(\theta)} + \left( \frac{1 - {n^{2} \cdot {{Sin}^{2}(\theta)}}}{n^{2}} \right)^{1/2}} = {f(\theta)}}}{and}} & (8) \\{{r(\theta)} = {\frac{f(\theta)}{1\; - \;{f(\theta)}} \cdot \frac{X}{\; n}}} & (9)\end{matrix}$

For small values of θ, f(θ)≅n/(n+1).

Thus, the length X may be selected so thatX≈R _(eq)/(n=1)where R_(eq) is some equivalent radius that is approximately equal tothe radius of the dome.

For specific deflection angle φ, the following equation may be solved:φ=θ+α=θ+Sin⁻³(nSin(θ))and θ=θ[φ] may be determined.

Therefore, the aspheric dome can be described by:ρ(φ)=Sin(φ)·r(θ[φ]) and  (10)Y(φ)=X−Cos(φ)·r(θ[φ])  (11)where ρ(φ) is the distance from the optical axis OA to a point on thedome 582 (as shown in FIG. 19) and Y(φ) is the distance along theoptical axis OA ordinate from the vertex of the dome to a point on thedome 582, as shown in FIG. 19.

Thus, a dome 582 may be constructed with a spherical surface of a singleradius that closely corresponds to the ideal structure derived above atevery point on the surface of the dome 582. It is appreciated that theincidence angle of a ray of light would then deviate slightly from thecentral wavelength of the interference filter but would remainsignificantly less than the variation resulting from a conventionalinterference filter. It also appreciated that if the dome has a lowoptical power, then the coating could be place either side of the dome,without significantly changing the optical paths of the light passingthrough the coatings, which comprise the optical filter.

Reference is now made to FIG. 20, a simplified flow chart illustratingoperation of a data entry object engagement location sensing subsystememployed in the projection keyboard system and methodology of FIG. 1 inaccordance with a preferred embodiment of the present invention and toFIG. 21, which is a simplified illustration of a preferred datastructure employed in the operation of the data entry object engagementlocation sensing subsystem shown in FIG. 20.

FIG. 20 shows a simplified illustration of a preferred data structureemployed in the operation of the data entry object engagement locationsensing method described hereinbelow with respect to FIG. 21. It isappreciated that the imaging sensor of a camera, such as camera 516(FIG. 17A), is typically comprised of a set of M×N pixels, wherein aparticular group of pixels views a defined region of the engagementplane which preferably overlies the projection surface 104 (FIG. 1).Thus, it is possible that a particular pixel group, located in the imageplane of the camera 516 may receive scattered light from a data entryobject 512 engaging the key location 113.

Thus, as the camera 516 views the projection surface 104, each of theM×N pixels in the image plane of the camera 516 may receive light from acorresponding region in the engagement plane in respect of a data entryobject engagement therewith.

Thus, as each pixel value is acquired, a determination is made, usingthe pixel coordinates, as to whether that pixel lies within a predefinedkeystroke region, such as keystroke regions 600 shown in FIG. 22. Thisdetermination is preferably made by employing a pixel index table 601which indicates for each pixel, whether that pixel lies within apredetermined keystroke region, such as keystroke regions 625, 626, 627and 628 (FIG. 22), and, if so, within which keystroke region it lies.

As seen in FIGS. 20 and 21, pixel values, such as gray level values, areacquired for various pixel coordinates. As each pixel value is acquired,a determination is made, using the pixel coordinates, as to whether thatpixel lies within a predefined keystroke region (FIG. 22). Thisdetermination is preferably made by employing a pixel index table 601which indicates for each pixel, whether that pixel lies within apredetermined keystroke region and, if so, within which keystroke regionit lies.

The function of the pixel within the keystroke region in which it liesis then determined, preferably by employing table 601. This function istypically additive or subtractive, but may alternatively have anotherfunction. Typically, depending on the function, the pixel value is addedto or subtracted from a pixel total maintained for each keystroke regionin a keystroke region accumulator table 602.

Once all of the pixels in a frame have been processed as aforesaid, anupdated background level is determined for the frame and a key actuationthreshold is determined typically by subtracting the updated backgroundlevel from a predetermined threshold level which is established for eachkeystroke region. This is preferably carried out by employing akeystroke region threshold table 604.

The contents of the keystroke region accumulator table 602 for eachkeystroke region preferably are then compared with the current keyactuation threshold. If the contents of the accumulator table 602 exceedthe key actuation threshold for a given key actuation region in a givenframe and in the previous frame the contents of the accumulator table602 did not exceed the key actuation threshold, a key actuation outputis provided.

Similarly, if the contents of the accumulator table 602 does not exceedthe key actuation threshold for a given key actuation region in a givenframe and in the previous frame the contents of the accumulator table602 did exceed the key actuation threshold, a key deactuation output isprovided. In all other cases, no output need be generated.

Reference is now made to FIG. 22, which is a simplified pictorialillustration of outlines of typical keystroke regions 625,626,627 and628 as sensed by a two-dimensional image sensor (FIG. 1) viewing akeyboard, such as the keyboard 190, seen in FIG. 5A.

Reference is now made to FIG. 23, which is a simplified pictorialillustration of outlines of typical footprints 629, 630, 631 and 632 ofa typical light pattern occasioned by data entry object engagementcorresponding to the keystroke regions 625,626,627 and 628 (FIG. 22).

Reference is now made to FIGS. 24A, 24B and 24C, which are simplifiedillustrations of three alternative methodologies for determining thefunction of the pixel within the keystroke region in which it lies asshown in FIG. 21 and to FIGS. 23A, 23B and 23C, which are simplifiedillustrations of traces which are useful in understanding FIGS. 22A, 22Band 22C.

Turning now to FIG. 24A, there is shown arranged along a commonarbitrary axis 610 a simplified keystroke region 620 corresponding to agiven key and containing a plurality of pixels 622. A typical simplifiedfootprint of a typical light pattern occasioned by data entry objectengagement with the given key is indicated by reference numeral 624.FIG. 23 shows outlines of typical footprints 625, 626, 627 and 628,corresponding to keystroke regions designated 629, 630, 631 and 632 inFIG. 22.

A typical background signal pattern is indicated by reference numeral634. Superimposition of the footprint 624 over the background signalpattern 626 is indicated at reference number 635. A one dimensionallyselectable overlap of footprint 624 over keystroke region 620 isindicated at reference numeral 636. A one dimensionally selectableoverlap of background signal pattern 634 over keystroke region 620 isindicated at reference numeral 637. A one dimensionally selectableoverlap of superimposition 638 over keystroke region 620 is indicated atreference numeral 638.

FIG. 25A illustrates a bias function 640 corresponding to a crosssection of the keystroke region 620 taken along axis 610, which biasfunction defines keystroke region 620 along axis 610. There is also seena signal function 644 corresponding to a cross section of the footprint624 along axis 610; a background signal function 646 corresponding to across section of the background signal pattern 634 along axis 610 and acombination signal 648 corresponding to a cross section of thesuperimposition 635.

There is also shown in FIG. 25A a one dimensional convolution 650corresponding to one dimensionally selectable overlap 636; a onedimensional convolution 652 corresponding to one dimensionallyselectable overlap 637 and a one dimensional convolution 654corresponding to one dimensionally selectable overlap 638. Convolution650 is shown with a threshold 660; convolution 652 is shown with athreshold 662 and convolution 654 is shown with a threshold 664.

Turning now to FIG. 24B, there is shown arranged along a commonarbitrary axis 670 a simplified keystroke region 680 corresponding to agiven key and containing a plurality of pixels 682. The simplifiedkeystroke region 680 is here shown surrounded by a simplified keystrokeregion border 683. A typical simplified footprint of a typical lightpattern occasioned by data entry object engagement with the given key isindicated by reference numeral 684. A typical background signal patternis indicated by reference numeral 686. Superimposition of the footprint684 over the background signal pattern 686 is indicated at referencenumeral 688. A one dimensionally selectable overlap of footprint 684over keystroke region 680 and border 683 is indicated at referencenumeral 690. A one dimensionally selectable overlap of background signalpattern 686 over keystroke region 680 and border 683 is indicated atreference numeral 692. A one dimensionally selectable overlap ofsuperimposition 688 over keystroke region 680 and border 683 isindicated at reference numeral

FIG. 25B illustrates a bias function 700 corresponding to a crosssection of the keystroke region 680 and of the border 683 taken alongaxis 670, which bias function defines keystroke region 680 along axis670. It is seen that border 683 is assigned a negative value relative tothe value of the keystroke region 680. This value assignment isappreciated to enhance the value of data entry object engagements to theextent that they lie within the keystroke region 680 and to decrease thevalue of such data entry object engagements to the extent that theyextend over the border 683. There is also seen a signal function 704corresponding to a cross section of the footprint 684 along axis 670; abackground signal function 706 corresponding to a cross section of thebackground signal pattern 686 along axis 670 and a combination signal708 corresponding to a cross section of the superimposition 688.

There is also shown in FIG. 25B a one dimensional convolution 720,corresponding to one dimensionally selectable overlap 690; a onedimensional convolution 722, corresponding to one dimensionallyselectable overlap 692 and a one dimensional convolution 724corresponding to one dimensionally selectable overlap 694. Convolution720 is shown with a threshold 726; convolution 722 is shown with athreshold 727 and convolution 724 is shown with a threshold 728.

Turning now to FIG. 24C, there is shown arranged along a commonarbitrary axis 730 a simplified keystroke region 740 corresponding to agiven key and containing a plurality of pixels 741. The simplifiedkeystroke region 740 is here shown surrounded by a simplified keystrokeregion inner border 742 and by a simplified keystroke region outerborder 743. A typical simplified footprint of a typical light patternoccasioned by data entry object engagement with the given key isindicated by reference numeral 744. A typical background signal patternis indicated by reference numeral 746. Superimposition of the footprint744 over the background signal pattern 746 is indicated at referencenumeral 748. A one dimensionally selectable overlap of footprint 744over keystroke region 740 and borders 742 and 743 is indicated atreference numeral 750. A one dimensionally selectable overlap ofbackground signal pattern 746 over keystroke region 740 and borders 742and 743 is indicated at reference numeral 752. A one dimensionallyselectable overlap of superimposition 748 over keystroke region 740 andborders 742 and 743 is indicated at reference numeral 754.

FIG. 25C illustrates a bias function 760 corresponding to a crosssection of the keystroke region 740 and of the borders 742 and 743 takenalong axis 730, which bias function defines keystroke region 740 alongaxis 730. It is seen that border 742 is assigned a zero value and border743 is assigned a negative value relative to the value of the keystrokeregion 740. It is appreciated that these value assignments enhance thevalue of data entry object engagements that to the extent that they liewithin the keystroke region 740 and to decrease the value of such dataentry object engagements to the extent that they extend across theborder 742 and at least into border 743. There is also seen a signalfunction 764 corresponding to a cross section of the footprint 744 alongaxis 730; a background signal function 766 corresponding to a crosssection of the background signal pattern 746 along axis 730 and acombination signal 768 corresponding to a cross section of thesuperimposition 748.

There is also shown in FIG. 25C a one dimensional convolution 780,corresponding to one dimensionally selectable overlap 750; a onedimensional convolution 782, corresponding to one dimensionallyselectable overlap 752 and a one dimensional convolution 784corresponding to one dimensionally selectable overlap 754. Convolution780 is shown with a threshold 786; convolution 782 is shown with athreshold 787 and convolution 784 is shown with a threshold 788.

It may be appreciated from a consideration of convolutions 638, 694 and754 that the dual border arrangement of FIGS. 24C and 25C provides thebest detection of data entry object keystroke engagement, whileminimizing background effects.

Reference is now made to FIG. 26, which is a simplified flow chartillustrating operation of a data entry object engagement locationsensing subsystem employed in a tracking system and methodologyconstructed and operative in accordance with a preferred embodiment ofthe present invention.

As seen in FIG. 26, pixel values, such as gray level values, areacquired for various pixel coordinates. As each pixel value is acquired,a determination may be made, using the pixel coordinates, as to whetherthat pixel lies within a predefined active region. Typically, if thepixel does not lie within a predetermined active region, its value isignored.

The pixel value for each pixel is preferably thresholded and typicallyall pixel values falling below a given threshold are ignored. Theremaining pixel values may be weighted by a selected weightingparameter. In order to determine the “center of gravity” of thethresholded and weighted pixel values, the thresholded and weightedpixel values are multiplied respectively by X and Y values representingthe coordinate position of each pixel and the results are summed alongmutually perpendicular axes X and Y and stored in X and Y accumulators.The total of the thresholded and weighted pixel values for all relevantpixels are also summed and stored in a data accumulator, for the entireactive region.

Once all of the pixels in a frame have been processed as aforesaid, thesummed thresholded and weighted pixel values along the X and Y axesrespectively are divided by the total of the thresholded and weightedpixel values for the entire active region to determine the X and Ycoordinates of the “center of gravity”, which represents a desiredengagement location.

Reference is now made to FIG. 27, which is a simplified flowchartillustrating operation of functionality providing shadow sharpnessanalysis in accordance with a preferred embodiment of the presentinvention, to FIG. 28, which is a simplified illustration of a preferreddata structure employed in the operation of the data entry objectengagement location sensing subsystem shown in FIG. 27 and to FIG. 29,which is an illustration which is useful in understanding the flowchartof FIG. 27.

As seen in FIGS. 27-29, pixel values, such as gray level values, areacquired for various pixel coordinates. As each pixel value is acquired,a determination is made, using the pixel coordinates, as to whether thatpixel lies within a predefined keystroke region 800 (FIG. 29) andwhether it lies within left or right subregions 802 and 804respectively. This determination is preferably made by employing a pixelindex table 806 which indicates for each pixel, whether that pixel lieswithin a predetermined keystroke region and, if so, within whichkeystroke region as well as within which keystroke subregion it lies.

The derivative of the pixel values along the X axis 808 (FIG. 29) iscalculated and thresholded. X axis derivative values, the absolutevalues of which exceed a predetermined threshold, are summed for eachsubregion of each keystroke region and stored in a keystroke regionaccumulator table 810. The variation of pixel values along the X axis808 for a situation, such as that illustrated in FIG. 11A, is shown atreference numeral 812. The X-axis derivative thereof is shown atreference numeral 814. The variation of pixel values along the X axis808 for a situation, such as that illustrated in FIG. 11B, is shown atreference numeral 816. The X-axis derivative thereof is shown atreference numeral 818. The threshold applied to derivatives 814 and 818is indicated by reference numeral 820.

It is clearly seen that the closer that the data entry object is to theengagement surface 104 (FIGS. 11A & 11B), the sharper is the detectededge and the greater is the derivative.

Once all of the pixels in a frame have been processed as aforesaid a keyactuation threshold is determined typically from a predeterminedthreshold level which is established for each keystroke region. This ispreferably carried out by employing a keystroke region threshold table822.

The contents of the keystroke region accumulator table 810 for the twosubregions in each keystroke region preferably are then subtracted onefrom the other. The difference is compared with the current keyactuation threshold. If the difference exceeds a key actuation thresholdfor a given key actuation region in a given frame and in the previousframe the difference did not exceed the key actuation threshold, a keyactuation output is provided.

Similarly, if the difference does not exceed the key actuation thresholdfor a give key actuation region in a give frame and in the previousframe the difference did exceed the key actuation threshold, a keydeactuation output is provided. In all other cases, no output need begenerated.

Reference is now made to FIG. 30, which is a simplified illustrationshowing synchronized illumination power variation functionality usefulin accordance with a preferred embodiment of the present invention. Thefunctionality illustrated in FIG. 30 is directed to modulating theamount of illumination provided for data entry object engagementdetection. This modulation is desirable because the intensity of lightimpinging on a data entry object and is thus scattered thereby,decreases with the distance between an illuminator 830 and a data entryobject. Thus it may be appreciated that the amount of light impinging ona data entry 832 is substantially greater than the amount of lightimpinging on a data entry object 834. Furthermore the amount ofscattered light impinging on a detector 836 decreases with the distancebetween the data entry object and the detector. These two distancedependencies make detection of data entry object engagement difficult.

In order to overcome this difficulty, there is provided in accordancewith a preferred embodiment of the present invention variable intensitydrive electronics 840 which is coupled to both illuminator 830 anddetector 836 and which causes the intensity of light produced by theilluminator 830 to vary, typically in a ramp fashion, in synchronizationto the imaging field location of detector 836.

Thus, it may be seen that when a near portion (A) of the engagementsurface 104 (FIG. 1) is being imaged, typically at the top portion A ofdetector 836, the intensity is at a minimum. When an intermediateportion (B) of the engagement surface 104 is being imaged, typically atthe middle of detector 836, the intensity is at an intermediate leveland when a far portion (C) of the engagement surface 104 is beingimaged, typically at the bottom portion (C) of the detector 836, theintensity is at a maximum.

Variable intensity drive electronics 840 operates preferably byproviding a synchronization output 842 to detector 836 and acorresponding synchronization output 844 to illuminator 830, causing theintensity level to increase in time in synchronization with the locationof a scanned image region in detector 836.

Reference is now made to FIG. 31, which is a simplified illustration ofa system and functionality for providing a digital signature inaccordance with a preferred embodiment of the present invention. As seenin FIG. 29, an output from a data entry object engagement detectionsubsystem 850, such as detector subsystem 112 (FIG. 1), providesintensity, position and timing outputs which are combined in a digitalsignature generator 852. Digital signature generator 852 preferableprovides a unique digital signature based on these outputs. Theintensity and timing outputs may be generated by the functionalitydescribed hereinabove with reference to FIGS. 20 and 21. The positionoutput may be generated by the functionality described hereinabove withreference to FIG. 26.

Reference is now made to FIG. 32, which is a simplified illustration ofa keyboard system and methodology, constructed and operative inaccordance with a preferred embodiment of the present invention andemploying sensing of data entry object interaction with an inertkeyboard defined on a surface, such as a pull-down tray 900. Such apull-down tray may be located in a vehicle, such as an airplane, and mayhave multiple uses, such as a dining tray. The keyboard may be definedby printing on the tray or on a sheet which can be placed on the tray oralternatively by suitable illumination thereof. Data entry objectengagement detection may be provided by apparatus 902 incorporated inthe vehicle or alternatively by portable apparatus, such as that carriedby a passenger. Computer functionality may be provided by apparatusincorporated in the vehicle or alternatively by portable apparatuscarried by a passenger. Computer memory, such as a memory element 904,may be carried by a passenger and may be inserted into a suitable socket906 in the vehicle.

Reference is now made to FIG. 33, which is a simplified illustration ofa keyboard system and methodology, constructed and operative inaccordance with a preferred embodiment of the present invention andproviding alphanumeric annotation of photographs using a suitableequipped camera, such as a video camera 910. A keyboard 912 may beprojected by a projection subsystem 914 integrally formed or otherwiseassociated with camera 910 and data entry object engagement detectionmay be provided by detection apparatus 916, also integrally formed orotherwise associated with camera 910. The keyboard may advantageously beemployed for annotating pictures taken with the camera.

Reference is now made to FIGS. 34A, 34B, 34C and 34D, which aresimplified illustration of four alternative embodiments of a keyboardsystem and methodology, constructed and operative in accordance with apreferred embodiment of the present invention and providing control, bydata entry object interaction, of a home entertainment system. FIG. 34Ashows a keyboard 920 defined on a television screen, typically either byoperation of the television or by projection on the screen. Data entryobject engagement detection is provided by apparatus 922 which may beportable or fixedly attached to the television.

FIG. 34B shows a keyboard 930 defined alongside a home entertainmentsystem. The keyboard 930 may be provided by projection or may be printedonto any suitable surface. Data entry object engagement detection isprovided by apparatus 932 which may be portable or fixedly attached tothe home entertainment system.

FIG. 34C shows a user interface board 934 defined on a table alongside ahome entertainment system. The user interface board 934 may be providedby projection or may be printed onto any suitable surface. Data entryobject engagement detection is provided by apparatus 936 which may beportable or fixedly attached to the home entertainment system.

FIG. 34D shows a user interface board 938 defined on a remote controlunit alongside a home entertainment system. The user interface board 938may be provided by projection or may be printed onto any suitablesurface. Data entry object engagement detection is provided by apparatus939 which may be integralled formed of fixedly attached to the remotecontrol unit.

In all of the above embodiments, the keyboard can be used for anysuitable function, such as interactive entertainment and infotainment.

Reference is now made to FIG. 35, which is simplified illustration of arestricted particulate matter environment keyboard system andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention. A keyboard 940 may be provided byprojection or may be printed onto any suitable surface. Data entryobject engagement detection is provided by apparatus 942 which may beportable or fixedly attached to equipment. The keyboard 940 may beemployed for controlling the equipment.

Reference is now made to FIG. 36, which is a simplified illustration ofa industrial environment keyboard system and methodology, constructedand operative in accordance with a preferred embodiment of the presentinvention. A keyboard 950 may be provided by projection or may beprinted onto any suitable surface. Data entry object engagementdetection is provided by apparatus 952 which may be portable or fixedlyattached to industrial equipment. The keyboard 950 may be employed forcontrolling the industrial equipment.

Reference is now made to FIG. 37 which illustrate a video projector 960having integrally formed or associated therewith a keyboard system andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention. A keyboard 962 is preferablyprovided by projection or may be printed onto any suitable surface. Dataentry object engagement detection is provided by apparatus 964 which maybe portable or fixedly attached to the projector 960.

Reference is now made to FIG. 38, which is a simplified illustration ofa restaurant patron interface system and methodology, constructed andoperative in accordance with a preferred embodiment of the presentinvention. As seen in FIG. 38, a menu selection board 970 may beprovided by projection or may be printed onto any suitable surface. Dataentry object engagement detection is provided by apparatus 972 which maybe portable or fixedly attached to a table. A virtual credit cardsignature pad 974 may also be provided by projection or otherwise.Detection of a signature may also be provided by engagement detectionapparatus 972.

Reference is now made to FIG. 39, which is a simplified illustration ofa keyboard system and methodology, constructed and operative inaccordance with a preferred embodiment of the present invention andproviding alphanumeric annotation or photographs using a suitablyequipped audio player 980. A keyboard 982 may be projected by aprojection subsystem 984 integrally formed or otherwise associated withplayer 980 and data entry object engagement detection may be provided bydetection apparatus 986, also integrally formed or otherwise associatedwith player 980. The keyboard 982 may advantageously be employed forannotating or selecting music to be played by the player.

Reference is now made to FIGS. 40A and 40B, which are simplifiedillustrations of a data entry object engagement sensing screen systemand methodology, constructed and operative in accordance with apreferred embodiment of the present invention and which provides “touchscreen” functionality using data entry object engagement detectionfunctionality of the type described hereinabove. Data entry objectengagement with a screen 1000, a conventional CRT screen, a flat panelscreen, or a screen projected, in a manner similar to that of thevarious keyboards described hereinabove, may be detected by detectionapparatus 1002, integrally formed or otherwise associated with screen1000. The screen 1000 may be employed for any suitable application, suchas in an interactive information kiosk, one example of which is anautomatic teller machine.

Reference is now made to FIG. 41, which is a simplified illustration ofa security and access control system employing data entry objectengagement sensing methodology, constructed and operative in accordancewith a preferred embodiment of the present invention. Data entry objectengagement with a screen 1010, such as a conventional CRT screen, a flatpanel screen, or a screen, projected in a manner similar to that of thevarious keyboards described hereinabove, may be detected by detectionapparatus 1012, integrally formed or otherwise associated with screen1010. The screen 1010 may be located at any suitable location andemployed for entry of access information by a user.

Reference is now made to FIG. 42, which is a simplified illustration ofa object engagement sensing game system and methodology, constructed andoperative in accordance with a preferred embodiment of the presentinvention using data entry object engagement detection functionality ofthe type described hereinabove. Object engagement with a game board1020, which may be defined, for example by a conventional CRT screen, aflat panel screen, or a screen projected in a manner similar to that ofthe various keyboards described hereinabove, may be detected bydetection apparatus 1022, integrally formed or otherwise associated withgame board 1020. The game board 1020 and associated functionality may beemployed for any suitable game application, such as chess or checkers.

Reference is now made to FIGS. 43A, 43B and 43C, which are simplifiedillustrations of a data entry engagement sensing musical instrument andmethodology, constructed and operative in accordance with a preferredembodiment of the present invention using data entry object engagementdetection functionality of the type described hereinabove. Data entryobject engagement with piano keys 1030, drum surfaces 1032 and guitarfrettes 1034, which may be projected in a manner similar to that of thevarious keyboards described hereinabove, or otherwise defined, as bydrawing, may be detected by detection apparatus 1036. This embodiment ofthe present invention may be employed for any suitable hand operatedmusical instrument.

Reference is now made to FIG. 44, which is a simplified illustration ofa vehicle mounted user interface system and methodology, constructed andoperative in accordance with a preferred embodiment of the presentinvention. The system of FIG. 44 preferably projects a keyboard 1040onto a vehicle surface, preferably a vehicle windscreen. This keyboardmay be used for inputting information for any purpose, preferably forentering a desired destination into a navigation system. Data entryobject engagement with keyboard 1040, such as a conventional CRT screen,a flat panel screen, or a screen projected in a manner similar to thatof the various keyboards described hereinabove, may be detected bydetection apparatus 1042, integrally formed or otherwise associated withthe vehicle. The keyboard 1040 may be located at any suitable location.

Reference is now made to FIG. 45 which is a simplified illustration of avending machine incorporating a data entry object engagement detectionsystem and methodology, constructed and operative in accordance with apreferred embodiment of the present invention. Data entry objectengagement with selection board 1050, such as a conventional CRT screen,a flat panel screen, or a screen projected in a manner similar to thatof the various keyboards described hereinabove, may be detected bydetection apparatus 1052, integrally formed or otherwise associated withthe vending machine. The selection board 1050 may be employed for anysuitable user interaction with the vending machine, including not onlyselection of products, but also entry of payment information.

It will be appreciated by persons skilled in the art that the presentinvention is not limited by what has been particularly shown anddescribed hereinabove. Rather the scope of the present inventionincludes both combinations and subcombinations of various featuresdescribed hereinabove as well as variations and modifications thereofwhich do not form part of the prior art.

1. A data input device comprising: an illuminator operative toilluminate at least one engagement surface; a two-dimensional imagingsensor viewing said at least one engagement surface from a locationoutside said at least one engagement surface for sensing engagement of adata entry object with said at least one engagement surface; and a dataentry processor receiving an output from said two-dimensional imagingsensor and providing a data entry input to utilization circuitry, saiddata entry processor employing shadow analysis, said data entryprocessor comprising the following functionality: as each pixel value isacquired, determining, using the pixel coordinates, whether that pixellies within a predefined keystroke region and within predefined left andright keystroke subregions therewithin; acquiring pixel values forvarious pixel coordinates; obtainining the derivative of each pixelvalue along an X axis; summing said derivatives for each said subregion;subtracting the summed derivatives for the two subregions in eachkeystroke region, one from the other to provide a difference; comparingsaid difference with a current key actuation threshold; if thedifference exceeds the key actuation threshold for a given keystrokeregion in a given frame and in the previous frame the pixel total didnot exceed the key actuation threshold for that keystroke region,providing a key actuation output; and if the difference does not exceedthe key actuation threshold for a given keystroke region in a givenframe and in the previous frame the pixel total did exceed the keyactuation threshold for that keystroke region, providing a keydeactuation output.
 2. A data input device according to claim 1 andwherein: said illuminator comprises a non-point light source; and saiddata entry processor employs a shadow density analyzer to determine thesharpness of edges of a shadow defined by said non-point light sourceand said data entry object on said at least one engagement surface,which indicated the propinquity of the data entry object to said atleast one engagement surface.
 3. A data input device according to claim1 and wherein: said illuminator comprises a plurality of light sources;and said data entry processor employs a shadow coalescence analyzer todetermine the extent of coalescence of shadows defined by said pluralityof light sources and data entry object on said at least one engagementsurface, which indicated the propinquity of the data entry object tosaid at least one engagement surface.
 4. A data input device accordingto claim 1 and wherein said determining employs a pixel index tablewhich indicates for each pixel, whether that pixel lies within apredetermined keystroke region and, if so, within which keystroke regionas well as within which keystroke subregion it lies.
 5. A data inputdevice according to claim 4 and wherein said pixel total is maintainedfor each keystroke subregion in a keystroke subregion accumulator table.6. A data input device according to claim 1 and wherein said pixel totalis maintained for each keystroke subregion in a keystroke subregionaccumulator table.
 7. A data input device according to claim 1 andwherein said comparing employs a keystroke region threshold table.
 8. Adata input device according to claim 1 and wherein at least said atleast one engagement surface is associated with a pull-down tray in avehicle.
 9. A data input device according to claim 8 and wherein saidpull-down tray defined and engagement surface which is configured byprojection.
 10. A data input device according to claim 1 and wherein atleast said two-dimensional imaging sensor and said illuminator areassociated with a camera.
 11. A data input device according to claim 1and wherein at least said two-dimensional imaging sensor detector andilluminator are associated with a home entertainment system.
 12. A datainput device according to claim 11 and wherein said at least oneengagement surface overlies a television screen forming part of saidhome entertainment system.
 13. A data input device according to claim 1and wherein at least said at least one engagement surface is associatedwith a table.
 14. A data input device according to claim 1 and whereinat least said at least one engagement surface is associated with aremote control device.
 15. A data input device according to claim 1 andwherein at least said at least one engagement surface is located withina restricted particulate matter environment.
 16. A data input deviceaccording to claim 1 and wherein at least said at least one engagementsurface is located within an industrial environment unsuitable for aconventional keyboard.
 17. A data input device according to claim 1 andwherein at least said two-dimensional imaging sensor and saidilluminator are associated with a video projector.
 18. A data inputdevice according to claim 1 and wherein at least said two-dimensionalimaging sensor and said illuminator are associated with a restaurantpatron interface system.
 19. A data input device according to claim 1and wherein at least said two-dimensional imaging sensor and saidilluminator are associated with a mobile audio player.
 20. A data inputdevice according to claim 1 and wherein at least said two-dimensionalimaging sensor and said illuminator provide touch screen functionality.21. A data input device according to claim 20 and wherein said touchscreen functionality employs a video display screen.
 22. A data inputdevice according to claim 1 and wherein at least said two-dimensionalimaging sensor and said illuminator provide access controlfunctionality.
 23. A data input device according to claim 1 and whereinat least said at least one engagement surface is associated with a gameboard.
 24. A data input device according to claim 23 and wherein saidgame board defines an engagement surface which is configured byprojection.
 25. A data input device according to claim 1 and wherein atleast said at least one engagement surface is associated with a musicalinstrument.
 26. A data input device according to claim 25 and whereinsaid musical instrument defines an engagement surface which isconfigured by projection.
 27. A data input device according to claim 1and wherein at least said two-dimensional imaging sensor and saidilluminator provide vehicle telematics functionality.
 28. A data inputdevice according to claim 27 and wherein said vehicle defines andengagement surface which is configured by projection.
 29. A data inputdevice according to claim 1 and wherein at least said two-dimensionalimaging sensor and said illuminator provide automatic vending userinterface functionality.
 30. A data input method comprising:illuminating at least one engagement surface; viewing said at least oneengagement surface with a two-dimensional image sensor form a locationoutside said at least one engagement surface for sensing engagement of adata entry object with said at least one engagement surface; andprocessing an output from said two-dimensional imaging sensor andproviding a data entry input to utilization circuitry, said processingemploying shadow analysis, said processing comprising; as each pixelvalue is acquired, determining, using the pixel coordinates, whetherthat pixel lies within a predefined keystroke region and withinpredefined left and right keystroke subregions therewithin; acquiringpixel values for various pixel coordinates; obtaining the derivative ofeach pixel value along an X axis; summing said derivatives for each saidsubregion; subtracting the summed derivatives for the two subregions ineach keystroke region, one from the other to provide a difference;comparing said difference with a current key actuation threshold; if thedifference exceeds the key actuation threshold for a given keystrokeregion in a given frame and in the previous frame the pixel total didnot exceed the key actuation threshold for that keystroke region,providing a key actuation output; and if the difference does not exceedthe key actuation threshold for a given keystroke region in a givenframe and in the previous frame the pixel total did exceed the keyactuation threshold for that keystroke region, providing a keydeactuation output.
 31. A data input method according to claim 30 andwherein: said illuminating comprises a non-point light source; and saidprocessing employs a shadow density analyzer to determine the sharpnessof edges of a shadow defined by said non-point light source and saiddata entry object on said at least one engagement surface, whichindicates the propinquity of the data entry object to said at least oneengagement surface.
 32. A data input method according to claim 30 andwherein: said illuminating comprises a plurality of light sources; andsaid processing employs a shadow coalescence analyzer to determine theextent of coalescence of shadows defined by said plurality of lightsources and data entry object on said at least one engagement surface,which indicates the propinquity of the data entry object to said atleast one engagement surface.
 33. A data input method according to claim30 and wherein said determining employs a pixel index table whichindicates for each pixel, whether that pixel lies within a predeterminedkeystroke region and, if so, within which keystroke region as well aswithin which keystroke subregion it lies.
 34. A data input methodaccording to claim 33 and wherein said pixel total is maintained foreach keystroke subregion in a keystroke subregion accumulator table. 35.A data input method according to claim 30 and wherein said pixel totalis maintained for each keystroke subregion in a keystroke subregionaccumulator table.
 36. A data input method according to claim 30 andwherein said comparing employs a keystroke region threshold table.
 37. Adata input method according to claim 30 and wherein at least said atleast one engagement surface is associated with a pull-down tray in avehicle.
 38. A data input method according to claim 37 and wherein saidpull-down tray defines an engagement surface which is configured byprojection.
 39. A data input method according to claim 30 and whereinsaid illuminating and said sensing are associated with a camera.
 40. Adata input method according to claim 30 and wherein said illuminatingand said sensing are associated with a home entertainment system.
 41. Adata input method according to claim 40 and wherein said at least oneengagement surface overlies a television screen forming part of saidhome entertainment system.
 42. A data input method according to claim 30and wherein at least said at least one engagement surface is associatedwith a table.
 43. A data input method according to claim 30 and whereinat least said at least one engagement surface is associated with aremote control method.
 44. A data input method according to claim 30 andwherein at least said at least one engagement surface is located withina restricted particulate matter environment.
 45. A data input methodaccording to claim 30 and wherein at least said at least one engagementsurface is located within an industrial environment unsuitable for aconventional keyboard.
 46. A data input method according to claim 30 andwherein said illuminating and said sensing are associated with a videoprojector.
 47. A data input method according to claim 30 and whereinsaid illuminating and said sensing are associated with a restaurantpatron interface system.
 48. A data input method according to claim 30and wherein said illuminating and said sensing are associated with amobile audio player.
 49. A data input method according to claim 30 andwherein said illuminating and said sensing provide touch screenfunctionality.
 50. A data input method according to claim 49 and whereinsaid touch screen functionality employs a video display screen.
 51. Adata input method according to claim 30 and wherein said illuminatingand said sensing provide access control functionality.
 52. A data inputmethod according to claim 30 and wherein at least said at least oneengagement surface is associated with a game board.
 53. A data inputmethod according to claim 52 and wherein said game board defines anengagement surface which is configured by projection.
 54. A data inputmethod according to claim 30 and wherein at least said at least oneengagement surface is associated with a musical instrument.
 55. A datainput method according to claim 54 and wherein said musical instrumentdefines an engagement surface which is configured by projection.
 56. Adata input method according to claim 30 and wherein said illuminatingand said sensing provide vehicle telematics functionality.
 57. A datainput method according to claim 56 and wherein said vehicle defines anengagement surface which is configured by projection.
 58. A data inputmethod according to claim 30 and wherein said illuminating and saidsensing provide automatic vending user interface functionality.